Motion picture films are often affected by defects, i.e. undesirable objects such as scratches, dust, dirt, stains, abrasion etc. They usually originate from the technical process of developing, handling, storing, and screening or scanning the material. In some rare cases static objects may already be induced during capturing, for example fluff within a lens or dirt on a scanner glass. However, a very common defect is non-steady dirt, i.e. undesired objects that appear only for a single frame.
For archival and conservation purposes and for making use of the benefits of a digital representation, analogue motion picture films are scanned and digitally encoded. Restoration of the films can, therefore, be carried out in the digital domain after scanning. Instead of time consuming manual restoration of the digitized films by finding and removing each object, application of automatic restoration software with algorithms trying to detect and remove dirt objects is a cost saving alternative to manual workflow.
In a workflow for automatic film restoration, defective objects, such as dirt, can be detected automatically and the image region containing the detected defective object or blotch can be replaced by true image data, e.g. taken from another image frame of the sequence, such as for example a preceding image frame.
It is a common problem of automatic detection algorithms that defective objects remain undetected (false negative) or detected defective objects are wrongly detected, i.e. misdetected (false positive). Therefore, a semiautomatic restoration framework may provide a review step between detection and removal that allows an operator to deselect wrongly detected defective objects for preventing removal or mark undetected regions as defective objects. As an example, a restoration framework that organizes restoration related information in a metadata structure may provide a binary flag, e.g. named ‘remove’, for deselecting objects, which defaults to ‘true’ after detection of the potentially defective object and can be manually switched to ‘false’ within the review process.
Instead of performing a time consuming manual adjustment of the detection result, the amount of misdetections can be reduced prior to the review step by an automatic post-processing of the initial detection result, potentially superseding the manual adjustment.
P. M. B. von Roosmalen et al. noticed in “Restoration and Storage of Film and Video Archive Material” in Signal Processing for Multimedia, J. S. Byrnes (Ed.), IOS Press, 1999, that misdetections may occur due to noise and presented a probabilistic method for false alarm reduction.
In 2005, Attila Licsar et al. presented in “Trainable Post-Processing Method To Reduce False Alarms In The Detection of Small Blotches Of Archive Film”, IEEE International Conference on Image Processing, ICIP 2005, a blotch-analyzing (“post-processing”) step where the brightness (luminance channel) of the image is analyzed and regions are classified as blotch or as real object (no blotch). This classification is done by a feed-forward neural network trained by the image features of the detected blotches.
In 2006, D. Corrigan et al. pointed out in “Pathological Motion Detection for Robust Missing Data Treatment in Degraded Archived Media”, IEEE International Conference on Image Processing, ICIP 2006, that especially long-term pathological motion is a common source of falsely detected dirt. They presented a probabilistic framework using a five frames wide temporal window.
In 2007, S. Tilie et al. proposed in “A contrario False Alarms Removal for Improving Blotch Detection in Digitized Films Restoration”, 14th International Workshop on Systems, Signals and Image Processing, 2007 and 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, a high level method for false alarm detection that looks for “spatio-temporal alignment of detected pixels”.
However, these approaches are often either computationally intensive or lack accuracy.
There remains a need for a method and an apparatus for automatically determining, reliably and efficiently, whether a detection of an object detected in a frame of an image sequence as being defective, e.g. a potential dirt object, is a misdetection, i.e. a false alarm, thereby, e.g., avoiding that restoration of misdetected objects reduces a quality of a restored image sequence.